Explainable Machine Learning for Predicting Major Adverse Renal and Cardiovascular Events in Acute Myocardial Infarction With Chronic Kidney Disease: Algorithm Development and Validation

Introduction

Acute myocardial infarction (AMI) represents a major global public health challenge associated with significant mortality and disability. It is characterized by acute myocardial ischemia resulting from complete or incomplete occlusive thrombosis secondary to rupture or erosion of coronary atherosclerotic plaques [1]. Chronic kidney disease (CKD) is a strong independent risk factor for cardiovascular disease [2], affecting up to 30% to 40% of patients with AMI [3]. Compared with patients without CKD, those with AMI and comorbid CKD exhibit 2 to 5 times higher in-hospital mortality [4]. Reported rates of in-hospital major adverse cardiovascular events and acute kidney injury requiring dialysis range from 7.1% to 32.5% and 0.8% to 11.5%, respectively [5,6]. These poor outcomes are primarily attributed to frequent ineligibility for sustained reperfusion therapy and evidence-based pharmacotherapy, chronic volume overload, high comorbidity burden, and delayed medical attention due to atypical symptoms [3,5]. Consequently, patients with AMI and CKD impose a heavy socioeconomic burden and represent a critical population urgently requiring improved prognostic outcomes in cardiorenal management.

Recognizing the close cardiorenal interplay, the American Heart Association (AHA) has endorsed major adverse renal and cardiovascular events (MARCE), encompassing both cardiovascular and renal adverse events, as a novel composite clinical end point for evaluating cardiorenal disease prognosis [7]. This clinically significant end point is considered to provide a more comprehensive assessment of overall patient risk. Nevertheless, widely used traditional clinical prediction tools such as the Global Registry of Acute Coronary Events (GRACE) score and Cleveland Clinic score [8,9] exhibit critical limitations: they predominantly focus on single-organ end points (either cardiac or renal events) and fail to integrate the complexity of MARCE in cardiorenal comorbidity. More importantly, as approximately 75% of coronary heart disease-related clinical trials exclude patients with CKD [10,11], the applicability and predictive performance of existing models developed in “relatively healthier” populations are severely compromised when applied to real-world patients with AMI and prevalent CKD. Thus, there is an urgent clinical need for a robust novel tool to accurately predict MARCE risk in this high-risk population.

Machine learning (ML) methods demonstrate significant advantages in capturing complex nonlinear relationships and higher-order interactions within multidimensional data [12]. They have been extensively applied across medical research domains, including risk stratification and prognostic prediction [13,14]. Previous studies have demonstrated that ML-based models outperform traditional scoring systems in predicting adverse cardiovascular outcomes in patients with AMI [15,16]. However, widespread adoption of ML models in clinical decision support faces a major challenge: limited interpretability due to their inherent “black-box” nature [17,18], which hinders clinicians’ understanding and trust in model predictions. To address this limitation, this study integrates advanced ML algorithms with a Shapley additive explanations (SHAP)–based explainable AI framework. SHAP provides intuitive, individualized feature contribution explanations.

In this study, we aimed to develop and externally validate ML models for predicting in-hospital MARCE risk in patients with AMI and CKD. The models were designed to assist clinicians in identifying individual risk drivers, assessing disease severity, and supporting early risk stratification during index hospitalization.

Methods

Study Design

Based on predefined inclusion and exclusion criteria, this retrospective cohort study enrolled patients with AMI and CKD who presented and underwent emergency treatment at the First Affiliated Hospital of Wenzhou Medical University between January 2013 and July 2019. This group was designated the “discovery cohort” for model development and internal validation. An external validation cohort consisted of patients meeting the same criteria admitted between January 2020 and August 2021 at Wenzhou TCM (traditional Chinese medicine) Hospital of Zhejiang Chinese Medical University.

Inclusion criteria were as follows: (1) age 18 years or older; (2) diagnosis of AMI according to established guidelines [1]; (3) diagnosis of CKD, defined as an estimated glomerular filtration rate (eGFR) of <60 mL/min/1.73 m², calculated using the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) equation [19] from the first serum creatinine measured within 24 hours after admission; and (4) receipt of emergency treatment. Exclusion criteria included (1) death during intervention or within 24 hours of admission; (2) unavailability of coronary angiography images; (3) current dialysis or renal replacement therapy before admission; and (4) missing data exceeding 20% of individual patient records.

Ethical Considerations

The study protocol received approval from the ethics review boards of both the First Affiliated Hospital of Wenzhou Medical University (number R2152025) and Wenzhou TCM Hospital of Zhejiang Chinese Medical University (number WZY2024-KT-0294‐01) and conformed to the Declaration of Helsinki. The retrospective design exempted the requirement for participant consent. This study was reported in accordance with the TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) statement (Checklist 1) [20].

Data Collection

All data were extracted from the electronic medical record system within the first 24 hours of admission. The collected information included demographic characteristics, clinical presentation on admission, medical history, laboratory findings, echocardiography measurements, procedural details, and medication use.

Medical history, including hypertension, diabetes mellitus, and stroke, was identified based on International Statistical Classification of Diseases, Tenth Revision (ICD-10) codes. Laboratory results were obtained from tests conducted on the first day after admission. Echocardiography measurements were performed at admission, with left ventricular ejection fraction calculated using the modified Simpson biplane method from apical four-chamber and two-chamber views. Angiographic and procedural documentation included the degree and location of coronary stenosis, as well as the presence of calcification, thrombus, and whether thrombus aspiration was performed.

Medications administered during hospitalization, including beta-blockers, angiotensin-converting enzyme inhibitors (ACEIs)/angiotensin II receptor blockers (ARBs)/angiotensin receptor-neprilysin inhibitors (ARNIs), and diuretics, were also recorded.

Killip classification was defined as follows: class I indicated no signs of heart failure; class II was defined by the presence of rales and/or jugular venous distension; class III represented acute pulmonary edema; and class IV denoted cardiogenic shock [21].

The GRACE score, which is recommended by clinical guidelines for risk stratification in acute coronary syndrome [8,22], was calculated for each patient to enable comparison with the nomogram developed in this study.

End Point and Follow-Up

The primary end point was the occurrence of MARCE during hospitalization. MARCE was defined as a composite end point comprising all-cause mortality, acute or worsening renal failure requiring new renal replacement therapy (dialysis), new or recurrent AMI, and stroke [7]. Follow-up started at admission and ended at discharge or at in-hospital death.

Feature Selection and Data Preprocessing

The initial dataset comprised 65 clinical variables. Among these, 59 features with missing data ≤10% were retained and imputed using the Multivariate Imputation by Chained Equations (MICE) package (version 3.13.0; Stef van Buuren and Karin Groothuis-Oudshoorn) in R software (version 4.1.0; R Foundation for Statistical Computing), with predictive mean matching [23]. To identify the most relevant predictors, least absolute shrinkage and selection operator (LASSO) regression with 10-fold cross-validation was applied based on the minimum binomial deviance criterion in the training set [24].

Given the wide numerical range across features and the requirement of certain ML algorithms for normalized inputs, min-max scaling was applied to continuous variables, and multicategory variables were processed using 1-hot encoding.

Model Development

Two model types were constructed: a nomogram and tree-based ML models. The discovery cohort was randomly divided into a training set (375/536, 70%) and a test set (161/536, 30%). To ensure robustness and minimize bias, 10-fold cross-validation was applied during model training [25]. Each fold was created via stratified sampling to maintain the outcome distribution. Feature selection and model tuning were performed only on the training data, and final model evaluation was conducted on the untouched test set. Specifically, LASSO feature selection was performed once on the entire training set before cross-validation. The selected predictors were then used for subsequent model training, hyperparameter tuning, internal cross-validation, hold-out test evaluation, and external validation. The hold-out test set and external validation cohort were not used during feature selection or hyperparameter tuning.

ML Algorithms and Interpretation

We used decision tree (DT), random forest (RF), and extreme gradient boosting (XGBoost) as representative tree-based algorithms. DT segments the feature space via hierarchical splits but is prone to overfitting and ignores feature interactions. RF combines multiple trees via bagging and random feature selection to improve accuracy and generalizability [26]. XGBoost enhances performance through regularized gradient boosting, parallel processing, and efficient feature handling [27].

Seven predictors identified by LASSO (Killip class, lactate, eGFR, cardiogenic shock, beta-blockers, ACEI/ARB/ARNI, and diuretics) were examined in multivariable modeling. Because Killip class IV is clinically defined as cardiogenic shock, these 2 variables represent substantial conceptual overlap. To improve clinical interpretability and reduce redundancy, we retained Killip class as the integrated severity indicator in the final parsimonious model and did not retain cardiogenic shock as a separate predictor after adjustment in the logistic regression (LR). Therefore, the final 6-variable predictor set should be considered a clinically refined model based on LASSO-selected predictors rather than a model derived exclusively through a fully data-driven feature-selection process.

We then trained XGBoost, RF, DT, and LR models using the same 6 clinically interpretable predictors. To interpret model predictions, SHAP values were used to quantify feature importance and to visualize both global and local contributions [28].

Nomogram Construction

A nomogram was developed using multivariable LR based on the 6 selected features. To facilitate clinical application, an online calculator was created for visualization and risk estimation.

Model Evaluation

All models were evaluated using 10-fold cross-validation on the training set and tested on the hold-out test set. The nomogram was further validated in the external cohort. Discrimination was assessed using the area under the receiver operating characteristic curve (AUC). Additional metrics included the Matthews correlation coefficient, precision (positive predictive value), negative predictive value, F₁-score, accuracy, sensitivity, and specificity. Because these classification metrics are threshold-dependent, they were interpreted together with receiver operating characteristic and calibration performance. Calibration was assessed using the Brier score and visualized using calibration curves with 400 bootstrap samples. Decision curve analysis (DCA) and clinical impact curves were used to assess clinical utility and net benefit. For classification metrics (accuracy, F₁-score, sensitivity, specificity, positive predictive value, and negative predictive value), a fixed probability classification threshold of 0.50 was used across all models, with MARCE as the positive class.

Two post hoc sensitivity analyses were conducted. First, the XGBoost model was refitted using conventional learning-rate values of 0.05 and 0.10 to assess whether model performance was dependent on the extremely small automatically tuned learning rate. Second, XGBoost, RF, DT, and LR models were reconstructed using all 7 LASSO-selected predictors, including cardiogenic shock. All sensitivity analyses used the original training-test split, and final evaluation was performed in the untouched held-out test set.

Statistical Analysis

Continuous variables are described as means (SD) and categorical variables as frequencies and proportions. The clinical characteristics of patients were compared using the 2-tailed Student t test, Wilcoxon test, chi-square test, or Fisher exact probability method as appropriate. The evaluation indices of the ML models were compared using 1-way ANOVA with multiple comparisons of the least-significant difference. A 2-tailed P value <.05 was considered statistically significant. The statistical analyses were performed using R (version 4.1.0 [29]). AUC comparisons between models were additionally assessed using the DeLong test.

Results

Study Population

The study flowchart is shown in Figure 1. From January 2013 to July 2019, a total of 536 patients were enrolled in the discovery cohort. The mean (SD) of their age was 72.83 (10.31) years, and 372 (69.4%) of the participants were male. Of them, 18.1% (n=97) died in the hospital, 8.6% (n=46) had renal failure necessitating new renal replacement therapy, 3.2% (n=17) had a new or recurrent myocardial infarction (MI), and 1.7% (n=9) had a stroke. They were randomly assigned to a training set (n=375) or a test set (n=161). There were no significant differences in the patient characteristics of the 2 datasets (Table 1). The rates of MARCE were 24.8% (n=133) in the train and test sets. In both sets, differences in clinical characteristics between patients who had MARCE and those who did not are outlined in Table 2. Data from the validation set were obtained and analyzed on 157 patients collected from Wenzhou TCM Hospital of Zhejiang Chinese Medical University, including 35 (22.3%) cases of MARCE (Table S1 in Multimedia Appendix 1).

‎

Feature Selection

Among the 59 demographic details, comorbidities, laboratory results, echocardiography measurements, procedural details, and medication parameters, 7 significant predictors of MARCE were selected in the training set by the LASSO algorithm (Figure 2A and B): Killip class, lactate, eGFR, cardiogenic shock, beta-blockers, ACEI/ARB/ARNI, and diuretics. Given that Killip class IV clinically corresponds to cardiogenic shock, we retained Killip class as the broader, clinically interpretable severity variable and excluded cardiogenic shock from the final parsimonious nomogram to avoid redundant representation of overlapping information. The multivariable analysis of the 7 LASSO-selected predictors is presented in Table 3. After excluding cardiogenic shock due to clinical redundancy with Killip class, the final 6 predictors were used for model development.

‎

ML Models

Six selected variables were included in the following ML models: XGBoost, RF, DT, and LR. Figure 3A and Table 4 illustrate model performance in the test set. XGBoost achieved an AUC of 0.882 (95% CI 0.816‐0.947), LR 0.875 (95% CI 0.799‐0.950), RF 0.870 (95% CI 0.800‐0.940), and DT 0.814 (95% CI 0.720‐0.886). Given the overlap in confidence intervals, these findings indicate comparable discrimination among XGBoost, LR, and RF rather than a definitive ranking of model superiority. Ten-fold cross-validation results for AUC and accuracy in the training set are summarized in Table S2 in Multimedia Appendix 1. Calibration plots showed acceptable calibration for all models (Figure 3B, Table 4). Other evaluation indicators are summarized in Table 4. Confusion matrices for all 4 classifiers in the test set are shown in Table S3 in Multimedia Appendix 1. Pairwise AUC comparisons showed no statistically significant differences among XGBoost, LR, and RF (XGBoost vs LR: P=.89; XGBoost vs RF: P=.81; LR vs RF: P=.92). Comparisons against DT were also not significant (all P>.05; Table S4 in Multimedia Appendix 1).

‎

Post hoc sensitivity analyses were performed to assess the robustness of the XGBoost model and the final predictor set. Refitting XGBoost with conventional learning-rate values of 0.05 and 0.10 yielded similar discrimination compared with the originally tuned model, with AUCs of 0.876 and 0.870, respectively, vs 0.882 for the original model (Multimedia Appendix 2). Additionally, models reconstructed using all 7 LASSO-selected predictors, including cardiogenic shock, showed comparable performance to the final 6-variable models. These findings suggest that the main results were not materially dependent on the extremely small tuned XGBoost learning rate or on the exclusion of cardiogenic shock (Tables S5 and S6 in Multimedia Appendix 1).

SHAP Values Through Visualization

Comparisons among the 4 ML-based models for MARCE prediction showed that XGBoost achieved the numerically highest AUC in the test set, although its performance was comparable to LR and RF rather than being definitively superior. The final hyperparameters for all classifiers are provided in Table S7 in Multimedia Appendix 1.

We used SHAP values to provide feature attribution values for each variable within the XGBoost model. Figure 4A describes the SHAP summary plot for the 6 input variables in the training set. The y-axis displays the 6 variables, ranked in order of importance by their mean absolute SHAP values. The x-axis indicates the SHAP values associated with each variable and patient, which allowed the determination of whether a feature had a negative effect on the prediction toward a non-MARCE class or a positive effect on the prediction toward a MARCE class (Figure 4B).

‎

Beyond ranking feature importance, SHAP analysis provided a patient-level attribution of predicted MARCE risk. The Killip class had the largest mean absolute SHAP value, indicating that hemodynamic severity was the dominant contributor to the model output. Diuretic use, eGFR, and lactate also contributed substantially, suggesting that volume overload or heart failure treatment status, renal dysfunction, and metabolic stress jointly influenced MARCE risk prediction.

Establishment and External Validation of the Nomogram Model

The 6 selected predictors were incorporated to form a MARCE estimation nomogram (Figure 5). We compared the performance of the nomogram model with the GRACE score in the test set. The nomogram achieved an AUC of 0.875 (95% CI 0.800‐0.934), whereas the GRACE score achieved 0.765 (95% CI 0.675‐0.826) (Figure 6A and B). Both models showed acceptable calibration in the test set, with mean absolute error values of 0.047 and 0.045, respectively (Figure 6C and D). DCA suggested a potential net benefit of the nomogram across clinically relevant threshold ranges, rather than uniformly across all possible thresholds (Multimedia Appendix 3). In additional AUC comparison analysis, the nomogram showed higher discrimination than the GRACE score in this cohort (AUC difference=0.110, P=.03; Table S4 in Multimedia Appendix 1).

To further evaluate the nomogram, we analyzed an external validation cohort of 157 patients from Wenzhou TCM Hospital of Zhejiang Chinese Medical University. The nomogram displayed an AUC of 0.873 (95% CI 0.714‐0.926; Multimedia Appendix 4). Calibration plots with 400 bootstrap resampling confirmed agreement between predicted and observed risk (mean absolute error=0.042; Multimedia Appendix 4). DCA and clinical impact curve suggested potential clinical utility in the validation cohort (Multimedia Appendix 4), and these findings should be interpreted with caution because of the limited external sample size.

To facilitate clinical use, we developed a web-based interface for estimating the probability of in-hospital MARCE (Multimedia Appendix 5) [30].

‎

‎

Discussion

Principal Findings

In the present study, we applied LASSO-based feature screening and developed a series of models for in-hospital MARCE prediction in patients with AMI and CKD. We identified 6 selected predictors and observed moderate-to-good discrimination across ML and LR models (test set: AUC 0.814, 95% CI 0.720-0.886; AUC 0.882, 95% CI 0.816-0.947). SHAP analysis provided interpretable global and individual feature contributions. We also developed a nomogram and externally validated it, with acceptable discrimination and calibration. Given the close performance among XGBoost, LR, and RF, our results support a balanced emphasis on both predictive performance and clinical interpretability.

Previous research on adverse event prediction models has mostly focused on patients with AMI and excluded patients with CKD, thus reducing performance when applied to patients with CKD [10,11]. Troponin protein, which has been identified as a risk factor for adverse outcomes and is involved in many risk scores, can be cleared by the kidneys, thus further leading to the reduced predictive efficacy of such models in the population with CKD [31-33]. In a small cohort of 113 patients, Fu et al [34] developed a risk score using patient demographic and laboratory data to predict in-hospital mortality in patients with AMI and end-stage renal disease. The risk score calibrated well and performed better than the traditional GRACE scores, but the study population was relatively small and limited to patients with end-stage renal disease, not the patients with CKD, who are more common in the clinical setting. Zhang et al [35] reported a predictive model for major adverse cardiovascular events prediction in patients with CKD undergoing percutaneous coronary intervention. They incorporated selected risk factors into a nomogram, while the discrimination needed to be improved (AUC=0.612). Current American College of Cardiology/AHA guidelines recommend MARCE as a novel clinical end point for patients with cardiovascular disease and CKD, for more accurate risk stratification and providing new pathophysiological insights. Thus, a predictive model aiming at the MARCE is warranted, especially for patients with AMI and CKD.

ML is an important supporting technology for AI that allows computers to learn, analyze, and construct models from input data, and then output predictions for new samples. Combined with electronic health record systems, ML could help doctors predict risk in cardiovascular disease, therefore aiding in clinical decision-making for patients. Khera et al [36] developed an XGBoost model, a neural network, and a meta-classifier model with patients from the American College of Cardiology Chest Pain-MI Registry and found that the XGBoost algorithm improved prediction performance for high-risk individuals. Advanced ML methods were also used to improve the prediction of cardiac risk based on radiomic profiling of coronary artery perivascular adipose tissue. However, the computationally heavy, long analysis times required with the model limited the application, and the model needed to better characterize the association of perivascular adipose tissue with the coronary atherosclerosis phenotype [37]. These studies indicated that ML is a reliable, novel method for clinical practice.

The SHAP analysis provided information complementary to traditional regression-based interpretation. Whereas LR and nomogram models summarize average population-level associations, SHAP decomposes each individual XGBoost prediction into feature-specific contributions. Thus, even with a small and clinically intuitive predictor set, SHAP may help clinicians identify whether a high predicted MARCE risk is mainly driven by hemodynamic severity, impaired renal function, elevated lactate, or medication-related factors. However, SHAP values should not be interpreted as causal effects, particularly for medication-related predictors such as diuretics, beta-blockers, and ACEI/ARB/ARNI, which may reflect confounding by indication, treatment eligibility, contraindications, or disease severity. Additionally, these medication-related predictors may partly capture physician decision-making and local treatment practice rather than baseline patient risk alone. Therefore, although they may improve risk prediction within the present dataset, their contribution should be interpreted cautiously when considering model transportability and implementation in health care systems with different treatment strategies.

In this study, we integrated ML workflows with MICE imputation, LASSO feature selection, cross-validation, hyperparameter tuning, and SHAP interpretation. These methods jointly supported the prediction of in-hospital MARCE in patients with AMI and CKD while retaining clinical interpretability.

Our study represents an application of established ML and nomogram modeling approaches to MARCE prediction in patients with AMI and CKD, rather than a fundamentally new modeling framework. Using 6 variables available early during hospitalization, clinicians may estimate MARCE risk and support early risk stratification. The nomogram showed higher discrimination than GRACE in this dataset; however, this comparison should be interpreted cautiously because both tools were evaluated in the same cohort, and the end point definitions differ from the original GRACE development setting. Therefore, the primary value of this study lies in its focus on a clinically high-risk cardiorenal population, the use of MARCE as a composite end point, and the integration of model interpretability with a parsimonious clinical prediction tool.

Although LASSO initially identified 7 predictors, the final 6-variable model was refined by excluding cardiogenic shock because of its clinical overlap with Killip class IV. This decision reflects a trade-off between statistical optimization and clinical interpretability. A fully data-driven model may retain all statistically selected variables, but redundant predictors can reduce model parsimony and complicate clinical interpretation. Therefore, we prioritized a parsimonious predictor set that preserved the main prognostic information while improving interpretability and usability in clinical settings. The sensitivity analysis using all 7 LASSO-selected predictors showed comparable performance, suggesting that this clinical refinement did not materially affect the main findings. Additionally, the extremely small learning rate selected during automated XGBoost tuning may represent a boundary solution of the optimization process rather than a biologically or clinically meaningful optimum. Although sensitivity analyses using conventional learning rates of 0.05 and 0.10 yielded similar AUC values, this finding highlights a limitation of automated hyperparameter optimization in relatively small clinical datasets and should be interpreted with caution.

Several limitations should be considered. First, despite external validation, the study sample was regionally limited, and the external cohort was modest in size. The external cohort included 157 patients and 35 MARCE events, which may affect the precision and stability of validation estimates, particularly for more complex ML models. Therefore, the external validation should be regarded as preliminary rather than definitive confirmation of model generalizability. Further multicenter studies with larger sample sizes are required to confirm model transportability, calibration stability, and clinical utility across different health care settings. Second, the study modeled the composite MARCE end point rather than individual end point components. Some components, such as recurrent MI and stroke, occurred infrequently, limiting stable component-specific prediction. Third, patients who died within 24 hours were excluded because complete predictor information within the predefined 24-hour assessment window was often unavailable, which may introduce survivorship bias and limit generalizability to the most critically ill patients. Fourth, because LASSO feature selection was performed once on the entire training set before cross-validation rather than nested within each resampling fold, cross-validation estimates in the training set may still be slightly optimistic. However, final model performance was evaluated in an untouched hold-out test set and an independent external validation cohort. Fifth, this retrospective study had moderate class imbalance, with a MARCE rate of approximately 25%. We did not apply oversampling, undersampling, or class weighting in the primary analysis to preserve the observed event prevalence and calibration of predicted risk. Therefore, threshold-dependent metrics should be interpreted cautiously. Finally, this study focused on in-hospital MARCE only; longer-term outcomes after discharge require future investigation.

Conclusions

In this study, based on 6 important predictors, we developed 3 tree-based ML models and a nomogram for in-hospital MARCE risk prediction in patients with AMI and CKD. The models demonstrated clinically meaningful discrimination and may support early risk stratification, pending further multicenter prospective validation.